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FIELD  TEST  OF  A  COMPUTER-DRIVEN  TOOL  TO  MEASURE 
PSYCHOLOGICAL  CHARACTERISTICS  OF  AIRCREW 


INTRODUCTION 

The  US  Air  Force  (USAF)  expects  consistent  high  performance  from  aviators 
throughout  their  careers.  The  current  reduction  of  US  military  forces  affects  the 
caree!'*?  of  rated  aircrew.  Force  reductions  and  technological  advances  will  create 
a  leaner  USAF  that  will  have  to  do  more  with  less.  It  is  known  that  the  pressures 
of  downsizing  in  private  industry  increase  numerous  factors  that  lead  to  job 
dissatisfaction,  and  decrease  job  effectiveness  (16).  If  the  personal  qualities  of 
well-adapted  aviators  could  be  defined,  perhaps  training  could  enhance  pilot 
performance  and  retention  throughout  the  USAF. 

Psychological  information  about  consistent  performers  i-  hard  to  obtain,  because 
aviators  are  reluctant  to  offer  details  about  themselves.  Tnere  are  at  least  two 
reasons  for  this  problem.  First,  their  responses  on  clinical  tests  might  jeopardize 
their  flying  status.  Second,  the  USAF  has  no  standardized  psychological  testing 
process  to  generate  information.  Because  there  is  no  routine  procedure  to  gather 
information,  data  is  usually  gathered  from  select  groups.  These  groups  have 
been  grounded  aviators  seeking  waiver,  or  special  “high  interest”  groups  required 
to  undergo  medical  and  psychological  testing. 

Aviators  are  reluctant  to  report  any  functional  deficit.  Admitting  to  mental 
health  problems  is  a  risk — both  occupationally,  and  within  group  dynamics  (11). 
Several  studies  have  documented  greater  truthfulness  in  answering  personal 
questions  through  computer  query  (13,  24),  and  .self-administration  by  a  computer 
would  offer  privacy.  Computerization  also  offers  the  flexibility  of  a  consistent 
approach  at  any  testing  location,  without  needing  specialized  technical  support. 
Aviators’  responses,  obtained  anonymously,  may  very  well  encourage  both 
participation  and  truthfulness. 

Psychological  information  is  needed  from  a  truly  “normative"  group,  the  squadron. 
Although  normative  data  is  widely  sought  for  use  in  psycfjological  evaluations 
(23),  the  last  normative  data  derived  on  USAF  aviators  was  completed  by  Fine 
and  Hartman  (7)  in  the  early  1960’s.  This  data,  now  more  than  25  years  old, 
is  based  on  outdated  instruments,  and  on  a  select  group  of  aviators  that  fiew 
quite  different  aircraft.  In  the  last  10  years,  other  specialized  aviator  populations 
have  been  studied,  such  as  undergraduate  student  pilots  (19),  and  US  Army 
helicopter  pilots  (18). 

The  typical  military  flying  squadron  is  not  homogeneous.  It  has  aviators  that 
range  in  experience,  motivation,  and  personality.  There  has  been  no  research 
study  to  collect  psychological  information  covering  a  USAF  squadron  of  successful 
aircrew  who  are  at  different  points  in  their  military  and  aviation  careers.  Although 
Ashman  and  Telfer  (2)  reported  on  a  group  of  14  Royal  Australian  Air  Force 
(RAAF)  pilots  from  one  Mirage  squadron,  data  was  gathered  on  only  the  Edwards 
Personality  Questionnaire.  Using  this  approach  seemed  optimal  because  such  a 
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study  could  glean  information  about  a  varied  group  of  pilots  who  know  each  other 
well.  Information  could  be  collected  about  their  own  relative  strengths  and 
weaknesses,  and  about  the  squadron’s  top  performers. 

The  current  study  sought  to  learn  not  only  about  top  performers,  but  also 
about  long-term  performers  in  aviation.  The  expected  psychological  resilience  of 
military  aviators  is  a  proud  heritage  (1);  however,  psychiatric  disorders  occur  in 
this  population.  In  fact,  one  mission  of  the  Neuropsychiatry  Branch  of  the  USAF 
Aerospace  Medicine  Consultation  Service  (ACS)  of  Armstrong  Laboratory  (Brooks 
AFB,  TX)  has  been  to  consult  with  aircrew  who  have  had  mental  health  problems. 
Over  the  last  10  years,  more  than  280  pilots  with  psychiatric  diagnoses  have 
been  evaluated  for  return  to  flying  duties  (17).  If  this  battery  could  increase 
psychiatric  epidemiology  data,  more  specific  preventive  medicine  models  might 
improve  retention  in  this  population. 

We  chose  to  develop  a  new  approach  to  solving  the  problems  of  testing  a 
normative  military  aviator  sample;  anonymous,  self-administered,  computerized 
clinical  tests.  Test  selection,  and  adaptation  for  computer  administration  followed. 
In  addition,  a  peer  rating  survey  was  produced  to  identify  (anonymously)  top 
performers  in  the  squadron.  Each  selected  individual’s  personal  qualities  would 
also  be  recorded  on  this  form.  The  development  and  initial  field  trial  of  this  test 
battery  became  the  primary  goals  of  this  project. 


Development  of  the  Test  Battery 

The  subjects  in  this  study  completed  standard  psychological  tests,  and  an 
epidemiologic  survey  presented  through  notebook  computer/software  technology. 
The  following  tests  were  chosen: 

(a)  Minnesota  Multiphasic  Personality  Inventory-2  (MMPI-2) 

(b)  Diagnostic  Interview  Schedule  (C-DIS) 

(c)  Personal  Characteristics  Inventory  (PCI) 

(d)  Multi-Dimensional  Aptitude  Battery  (MAB) 

These  tests  offered  a  broad  approach  to  measure  different  psychological 
characteristics  of  aircrew.  The  MMPI-2  is  the  newest  version  of  the  MMPI,  a 
personality  test  that  has  become  the  most  frequently  administered  psychological 
test  (12).  It  is  used  at  the  USAF  ACS  to  evaluate  psychiatrically  grounded  aircrew 
who  request  a  waiver  to  return  to  flying  duties.  Occupational  norms  have  been 
developed  for  different  populations  (3,  8);  Butcher  (5)  reported  that  at  least  one 
US  airline  uses  the  MMPI  as  an  adjunctive  tool  in  employee  selection.  In  a 
counter-balanced,  repeated-measures  study  of  the  computerized  MMPI  versus  the 
standard  pencil  and  paper  presentation,  Honaker  et  al.  (9)  supported  the  software 
version  of  the  MMPI’s  equivalency.  While  not  specifically  tested,  the  equivalency 
of  the  computerized  MMPi-2  is  generally  accepted  (15). 

The  National  Institute  of  Mental  Health’s  (NIMH)  DIS  is  a  widely  used 
epidemiologic  survey  developed  by  Robins  and  Helzer  to  screen  the  general 
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population  for  the  prevalence  of  psychiatric  disorders  (20).  Up  to  now,  there 
have  been  no  attempts  to  define  the  prevalence  of  mental  health  disorders  in 
the  professional  aviator  population.  The  computerized  version  (C-DIS)  has  been 
validated  for  test-retest  reliability  compared  to  the  trained  interviewer  approach;  it 
has  good  correlation  in  most  diagnostic  areas  (3).  Acceptance  of  the  instrument 
as  a  self-administered  questionnaire  has  been  generally  good  (25). 

The  MAB  is  an  IQ  test  developed  by  Douglas  Jackson  (10)  that  has  a  high 
correlation  (.94-. 98)  with  the  WAIS-R.  In  general,  the  USAF  aviator  population 
has  been  noted  to  have  above  average  IQ  (7).  It  was  expected  that  IQ  might 
show  a  strong  correlation  to  “top  performer”  aviators.  Administered  in  ten  7-minute 
blocks  by  computer  with  the  use  of  an  accompanying  booklet,  the  computerized 
MAB  measures  verbal,  performance,  and  full  scale  IQ.  The  subtests  measure 
Information,  Comprehension,  Arithmetic,  Similarities,  Vocabulary,  Digit  Symbol, 
Picture  Completion,  Spatial,  Picture  Arrangement,  and  Object  Assembly. 

Helmreich  et  al.  (21)  developed  the  PCI  to  assess  “crt  ,  coordination  qualities” 
in  aviators.  Some  commercial  airline  corporations  use  it  for  pitot  selection  (6). 
It  consists  of  254  questions  with  a  Likert  scale  response  pattern.  Aircrew  responses 
are  categorized  into  ei-^ht  groups  ranging  from  the  “right  stuff  to  the  “wrong  stuff 
in  crew  coordination.  Already  widely  used  in  aerospace  operations,  it  collects 
data  more  specific  to  aviation  skills. 

The  overall  format  was  arranged  into  two  2.5-hour  blocks,  that  were  not  to 
be  taken  by  the  same  individual  on  the  same  day.  Battery  1  consisted  of 
Demographic  questions  (see  Table  1),  followed  by  the  MMPI-2  and  then  by  the 
PCI;  Battery  2  contained  the  MAB  followed  by  the  DIS.  Programming  solutions 
increased  test  security,  assured  test  order  and  also  prevented  duplicate  answers. 
To  confirm  confidentiality  of  answers,  each  subject’s  test  responses  were  recorded 
on  an  individual  3.5-in.  data  diskette  identified  externally  by  a  random  number. 
Test  responses  could  not  be  accessed  or  scored  without  the  computer  scoring 
modules,  which  were  unavailable  at  the  squadron. 

The  peer  survey  (see  Appendix  A)  was  given  to  each  pilot  with  Battery  2. 
Tne  aviators  were  asked  to  identify  the  top  three  pilots  (lead  and  two  wingmen) 
in  their  squadron  with  whom  they  would  fly  combat.  They  could  not  select 
themselves.  For  each  individual  they  chose,  they  also  rank  ordered  in  importance 
four  personal  qualities  from  the  list  in  Table  2.  These  characteristics  were  a 
modified  grouping  from  two  sources:  a  NASA  peer  survey  of  astronauts  (21), 
and  a  summary  of  ‘top  pilot’  characteristics  suggested  by  past  aces  (14).  This 
survey  provided  a  frequency  count  of  most  chosen  characteristics,  and  also  offered 
a  rank  order  of  personal  qualities  in  selected  pilots. 

After  testing  ended,  each  pilot’s  name  and  random  numbers  were  known  only 
to  a  disinterested  third  party  (trustee)  chosen  by  the  squadron.  Each  subject’s 
two  random  numbers  were  linked.  The  trustee  also  replaced  the  names  of  the 
top  performers  on  the  peer  survey  with  their  respective  random  numbers.  Once 
com.pleted,  the  key  and  the  nameless  surveys  could  be  used  to  compare  testing 
responses  and  personal  qualities  of  top  performers. 
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Table  1 .  Demographics  Questions 


Current  Military  Airframe  Hours:  <100, 101-500,  >500 

Total  Military  Hours:  <500,  501-1000,  1001-2000,  >2001 

<500,  501-1000,  1001-2000,  >2001 

Y  or  N 

Company  Grade  or  Field  Grade 

0, 1 , 2  or  more 
30  or  younger,  31-40,  41  or  older 
Education  level:  Undergraduate  Degree  or  Graduate  Degree 

Previous  Active  Duty  Time:  y  or  n 

Gender:  M  or  F 


Total  Civilian  Hours: 

Do  you  have  Combat  Hours: 
Rank: 

Number  of  Divorces: 

Age: 


Table  2.  Pilot  Characteristics 


General  Knowledge 

Job  Performance 

Stress  Tolerance 

Leadership 

Group  Cohesiveness 

Teamwork 

Personality 

Communication  Skills 
Aggressiveness 


METHOD 


Subjects 

Twenty-nine  (29/36=80%)  pilot  volunteers  from  a  Selfridge  Michigan  Air  National 
Guard  (ANG)  F-16  squadron  consisting  of  36  pilots  were  recruited  as  subjects. 
Computer  malfunctions  and  diskette  duplications  discovered  when  data  was  being 
scored  accounted  for  varying  levels  of  test  completion.  Twenty-nine  (100%)  pilots 
completed  the  peer  rating  survey  of  top  performers,  defining  characteristics  of 
their  selections.  Demographic  information  was  available  for  twenty-three 
(23/29=79%)  of  the  pilot  subjects.  Specific  demographic  information  was  not 
collected  to  ensure  the  subjects’  anonymity.  Age  groupings  for  the  known  pilots 
spread  from  under  30  years  old  to  more  than  41  years  old,  and  all  were  male. 
Fifteen  (15/23=65%)  had  previous  active  duty  time.  Distribution  was  nearly  equal 
between  field  (10/23=43%)  and  company  (13/23=57%)  grade  officers.  Five 
(5/23=22%)  had  combat  experience.  Flying  time  for  the  majority  (15/23=65%) 
was  less  than  100  civilian  hours;  an  equal  number  (15/23=65%)  had  more  than 
1000  military  hours.  Twenty  (20/23=87%)  had  less  than  500  hours  in  the  F-16. 
Figure  1  shows  the  variety  of  experience  in  flying  time  in  the  F-16  and  total 
military  flying  hours  in  the  squadron. 


Procedures 

it  was  imperative  to  protect  anonymity  in  the  pilots’  responses.  Therefore, 
the  key  that  linked  aviators’  names  and  random  numbers  was  secured  from  both 
squadron  members  and  investigators.  A  trustworthy  individual  not  connected  with 
the  armed  services  was  chosen  by  the  squadron  to  be  a  trustee.  This  person 
held  in  confidentiality  subjects’  names,  random  numbers,  and  chosen  top  performers 
from  the  peer  survey.  In  this  study,  a  physician  living  in  another  city  was  chosen 
by  the  squadron  to  be  the  trustee.  After  coding  the  last  survey,  the  trustee 
shredded  any  remaining  surveys,  name  sheets  or  any  key  linking  subjects’  names 
and  randc  n  numbers.  A  testing  proctor  supervised  the  use  of  the  computers, 
ana  prevented  two  test  sections  from  being  completed  by  any  one  individual  on 
the  same  day.  Data  collection  occurred  over  approximately  4  months,  covering 
4  active  duty  ANG  weekends. 

All  subjects  initially  signed  a  consent  form  which  described  the  experiment, 
the  ar.onymity  of  their  data,  and  which  acknowledged  their  right  to  terminate 
project  participation  at  any  time  (See  Appendix  B).  Each  pilot  chose  euner  a 
blue  or  a  white  3.5-inch  data  diskette  from  separate  storage  bins  corresponding 
to  the  test  battery  thay  were  completing.  The  pilot  took  the  test  battery  on  the 
notebook  computer  and  by  using  matched  random  numbers  on  peel-off  stickers, 
turned  in  the  data  diskette  and  sealed  his  name  and  peer  rating  in  an  envelope 
which  was  mailed  to  the  trustee.  A  second  set  of  matched  random  numbers  was 
used  for  the  second  test  battery.  The  trustee  matched  random  numbers  for  the 
two  separate  diskettes  and  destroyed  the  names  of  all  the  subjects  and  forwarded 
the  random  numbers  and  peer  ratings  to  the  investigators. 
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Figure  1.  Squadron  flying  hours/Variety  of  experience;  n=23 


Military  Hours,  Total 


Statistical  Analysis 


Descriptive  statistics  were  run  on  the  test  results,  peer  surveys  and  demographics 
in  this  “pilot”  project.  Peer  surveys  were  tabulated  for  the  frequency  of  chosen 
top  performers  and  their  qualities.  A  frequency  count  of  qualities  and  their  rank 
order  was  also  performed.  The  pilot  demographic  information  and  scale  scores 
from  the  psychometric  battery  were  analy2ed  with  descriptive  statistics  to  begin 
developing  norms.  Descriptive  statistics  were  tabulated  using  the  Statgraphics 
statistical  program. 


RESULTS 

Twenty-nine  pilots  (80%  of  the  squadron)  parti'-ipated  in  the  computerized 
testing  and  completed  the  peer  rating  survey  for  their  preferred  combat  four-ship 
formation.  Results  were  tabulated,  both  for  the  most  frequently  chosen  qualities, 
and  for  relative  importance  through  rank  ordering.  The  same  two  pilots  were 
chosen  most  frequently  for  both  lead  and  wing  positions,  one  receiving  eighteen 
(18/29=62%)  votes,  and  the  other  receiving  thirteen  (13/29=44%)  votes.  One 
individual  was  chosen  for  the  wing  position  seven  (7/58=12%)  times,  but  not  for 
the  lead  at  all.  Possibly,  he  lacked  the  upgrade  qualification,  or  he  had  the  most 
desirable  characteristics  to  fill  the  wingman  position.  After  these  three  pilots, 
votes  were  scattered  for  the  rest  of  the  aviators  by  votes  of  one  or  two. 
Nonparticipating  pilots  got  twenty-one  votes  (21/87=24%)  to  be  in  the  combat 
formation.  While  this  diminishes  the  exact  meaning  of  those  chosen,  the  qualities 
for  all  selected  aviators  are  known. 

Qualities  chosen  for  the  lead  and  the  two  wingmen  positions  showed  both 
similarities  and  differences.  Leadership,  job  performance,  aggressiveness,  and 
general  knowledge  were  the  characteristics  chosen  most  frequently  for  the  lead 
position.  Wingman  qualities  chosen  most  frequently  were  job  performance,  general 
knowledge,  teamwork;  there  was  a  tie  between  personality  and  aggressiveness. 
Frequency  of  choices  are  shown  in  Tables  3  and  4.  When  considering  rank 
order  (RO);  the  most  chosen  characteristic  for  lead  was,  in  RO  1,  leadership 
(41%);  in  RO  2,  leadership  (21%);  in  RO  3,  aggressiveness  (31%);  and  in  RO 
4,  general  knowledge  (24%)  [See  Table  5).  In  contrast  for  wingman,  most  chosen 
was,  in  RO  1,  job  performance  (31%);  in  RO  2,  job  performance  (21%);  in  RO 
3,  teamwork  (28%),  and  in  RO  4,  general  knowledge  (24%)  [See  Table  6].  Figures 
2  and  3  summarize  both  frequency  of  choice  and  rank  order  components. 

Scores  for  fifteen  pilots  (15/29=51%)  were  available  for  the  MAB.  Mean 
scores  were  Verbal  IQ  125,  Performance  IQ  127,  and  Full  Scale  IQ  127.  These 
scores  are  all  in  the  Superior  range  of  intellectual  functioning.  All  of  the  subtest 
means  were  elevated  at  least  one  standard  deviation  higher  than  the  genera! 
population  normative  sample  (10).  The  pilots  had  mean  scores  one  and  one-half 
standard  deviations  above  the  normal  population  sample  mean  on  three  subtests: 
Arithmetic,  Vocabulary,  and  Digit  Symbol.  These  subtests  respectively  measure 
mental  computation,  knowledge  of  word  meanings,  and  visual  memory  with  novel 
nonverbal  learning.  Two  more  subtests  (Information  and  Spatial)  had  means 
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between  one  standard  deviation  and  one  and  one-half  standard  deviations  above 
the  mean.  These  two  subtests  respectively  measure  genera!  fund  of  knowledge 
and  recognition  of  two-dimensional  rotated  figures.  MAB  scores  are  summarized 
in  Table  7. 

Scores  from  twenty-three  (23/29=79%)  MMPI-2  tests  were  obtained.  Mean 
scores  are  noted  in  Table  8.  Of  note  was  the  reduced  level  of  the  fake  good 
scale  in  this  population,  with  a  mean  score  of  60T.  The  clinical  profile  was  within 
normal  limits  with  expected  low  scores  on  complaints  about  physical  health 
(Hs=48T),  depressive  complaints  (D=43T),  acknowledging  stereotypical  gender  roles 
(Mf=41T),  and  comfort  in  social  settings  (Si=40T).  Higher  scale  means  were 
noted  on  denying  any  problems,  being  optimistic  (Hy=53T)  and  being  active, 
outgoing  and  energetic  (Ma=51T).  These  same  clinical  scores  are  commonly  seen 
in  aircrew  who  are  evaluated  at  the  ACS  Neuropsychiatry  Branch  (22). 

PCI  scores  in  twenty  pilots  (20/29=69%)  were  avaiu  ble.  The  pilots  scored 
in  the  following  clusters.  Six  (30%)  were  in  the  “Group  RD'  (Responses  Desired), 
scoring  extremely  high  on  the  positive  dimensions  and  extremely  low  on  the 
negative  dimensions.  Two  (10%)  more  scored  in  the  “Group  10"  (one  of  the  best 
profiles),  with  high  scores  on  overall  goal  seeking,  achievement  motivation,  and 
interpersonal  orientation.  Seven  (35%)  were  in  the  “Group  00"  and  did  not  display 
profiles  that  are  known  to  relate  to  either  positive  or  negative  crew  resource 
management  behavior  (“middle  of  the  road”).  One  (5%)  pilot  scored  in  the  “Group 
21,”  one  of  the  less  desirable  groups  with  high  goal  seeking  and  achievement 
motivation,  but  has  high  verbal  aggressiveness.  One  (5%)  pilot  was  in  “Group 
20”  which  is  referred  to  as  the  “wrong  stuff  because  of  a  low  interpersonal 
orientation.  The  remaining  three  (15%)  pilots  scored  in  “Group  30*  where  a 
relative  lack  of  goal  seeking,  achievement  motivation,  interpersonal  orientation, 
and  mastering  new  activities  is  noted.  This  information  is  summarized  in  Figure 
4. 

When  the  peer  ratings  were  compared  to  the  PCI  scores,  the  top  six  pilots 
picked  for  the  combat  four  ship  scored  in  the  Groups  RD,  30,  21  and  00 
respectively.  Two  of  the  three  pilots  picked  most  often  for  lead  scored  in  the 
best  group  (RD).  The  pilot  picked  for  lead  the  second  most  often  was  in  the  less 
desirable  group  21.  Of  the  twenty  pilots,  this  pilot  was  the  only  one  to  score 
in  this  group.  The  two  pilots  chosen  most  often  for  wingman  scored  in  the  middle 
groups  (Groups  30  and  00).  It  appears  that  it  takes  all  kinds  of  individuals  to 
get  the  mission  completed  in  spite  of  their  crew  resource  management  style.  This 
trend  is  summarized  in  Figure  5. 

C-DiS  data  was  obtained  from  five  pilots  (5/29=17%).  In  those  respondents, 
only  one  showed  a  psychiatric  diagnosis:  nicotine  dependence.  Computer 
instructions  in  the  initial  round  of  testing  unfortunately  confused  many  aviators 
who  erroneously  ended  testing  in  Battery  2  before  proceeding  with  the  C-DIS. 
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Table  3.  Lead  Characteristics 


* 


Quality 

Leadership 

Job  Performance 

Aggressiveness 

Knowledge 

Stress  Tolerance 

Personality 

Teamwork 

Communication 

Group  Cohesiveness 

*  n=29 


Lead  Total 

(%  of  1 1 6  choices) 

19.8 
17.2 
16.4 

13.8 

9.5 

8.6 
6.9 

4.3 

3.4 
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Table  4.  Wingman  Characteristics 


Quality 

Job  Performance 

Knowledge 

Teamwork 

Personality 

Leadership 

Aggressiveness 

Stress  Tolerance 

Group  Cohesiveness 

Communication 

*  n=29 


Wingman  Total 
(%  of  232  choices) 

19.1 
6.5 

13.9 

10.8 

10 

10 

9.1 

6.2 
4.3 
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Table  5.  Lead  Characteristics 
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Group  Cohesiveness 


Table  6.  Wingman  Characteristics 
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Ld  =  Leadership,  Pf= Performance,  Ag= Aggressiveness,  Kw= Knowledge,  St= Stress  Coping 
Ps= Personality,  Tm=Teamwork,  Cm=Communication,  Gp=Group  Cohesiveness 

Figure  2,  Lead  position/Personal  qualities  chosen;  n=29  (116  responses) 
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Pf= Performance,  Kw= Knowledge,  Tm=Teamwork,  Ps=Personality,  Ag=Aggressiveness 
Ld= Leadership,  St=Stress  Coping,  Gp=Group  Cohesiveness,  Cm=Communication 

Figure  3.  Wingman  position/personal  qualities  chosen;  n=29  (232  responses) 


Table  7.  MAB  Scores  * 


Scale 

Mean 

S.D. 

Information 

32.8 

3.4 

Comprehension 

24.4 

1.5 

Arithmetic 

17.4 

1.7 

Similarities 

29.5 

3.6 

Vocabulary 

36.7 

5.2 

Digit  Symbol 

28.5 

3.8 

Picture  Completion 

27.9 

2.8 

Spatial 

36.5 

5.2 

Picture  Arrangement 

12.9 

2.5 

Object  Assembly 

16.2 

2.9 

Verbal  Scale  IQ 

125.0 

6.4 

Performance  Scale  IQ 

127.1 

8.7 

Full  Scale  IQ 

127.3 

6.6 

Raw  Scores 

*  n  =  15 
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Table  8.  MMPI-2  Scores 


* 


Scale 

Raw  Scores 
Mean 

Raw  Scores 
S.D. 

T  Scores 
Mean 

(L)  Lie  Scale 

3.8 

1.9 

51 

(F)  Infrequency  Scale 

2.1 

1.9 

42 

(K)  Suppressor  Scale 

20.4 

3.0 

61 

Hypochondriasis 

1.7 

1.2 

48 

Depression 

15.3 

3.8 

43 

Hysteria 

22.2 

3.7 

53 

Psychopathic  Deviance 

14.8 

2.8 

50 

Masculinity-Femininity 

21.6 

4.4 

41 

Paranoia 

9.5 

2.2 

48 

Psychasthenia 

4.6 

3.0 

47 

Schizophrenia 

5.0 

3.4 

48 

Hypomania 

16.2 

4.2 

51 

Social  Introversion 

17.5 

5.9 

40 

*  n  -  23 
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Number  of  Pilots  chosen 


DISCUSSION 


The  major  goals  of  this  project  were  to  develop  and  field  test  a  computerized 
battery  to  gather  important  psychological  information  about  military  aviators.  To 
reduce  sampling  bias,  it  was  optimal  to  study  a  squadron  of  pilots  in  the  field. 
To  attain  the  goal  of  gathering  normative  informatioii  on  clinical  tests,  anonymity 
was  considered  crucial.  An  ANG  F-16  squadron  at  Selfridge  ANG  Base,  Michigan 
with  a  range  of  ages  and  flying  experience,  offered  to  participate  in  this  innovative 
program. 

This  project  considered  five  questions.  First,  what  instruments  should  be 
included  in  a  psychometric  test  battery  for  aircrew?  Second,  would  aviators 
volunteer  sensitive  information  about  themselves,  if  they  were  assured  of  anonymity? 
Third,  could  aviators  agree  on  their  squadron’s  top  performers?  Fourth,  could 
they  agree  on  the  qualities  important  in  those  chosen?  Fifth,  if  truly  normative 
data  were  gathered,  would  top  performers’  profiles  be  different? 

The  development  of  a  computerized  version  of  the  PCI  was  one  of  the  major 
accomplishments  of  this  project.  Along  with  this  measure  of  crew  coordination 
(PCI),  measurements  of  personality  (MMPI-2),  cognitive  capacity  (MAB);  and  one 
psychiatric  epidemiology  survey  (C-DIS)  were  combined  into  a  single  computerized 
protocol.  This  group  of  tests  queried  a  broad  range  of  mental  characteristics.  A 
tremendous  effort  was  needed  to  combine  ♦nur  different  “off  the  shelf  tests  into 
one  computer  presentation.  The  usefulness  was  demonstrated  by  making  this 
custom-designed  battery  of  tests  that  measured  intelligence,  personality,  crew 
coordination  attitudes,  and  presence/prevalence  of  psychiatric  diagnoses. 

Eighty  percent  of  this  squadron  participated  in  the  testing.  Maintaining  anonymity 
by  using  a  self-administered  computerized  format  was  another  major  success  in 
this  project.  It  revealed  that  pilots  would  voluntarily  offer  sensitive  information 
about  themselves  and  their  peers.  Twenty-nine  peer  surveys  (80%)  and  22 
MMPI-2  (61%),  20  PCI  (55%),  14  MAB  (38%),  and  5  DIS  (13%)  tests  were 
collected  from  the  squadron  pilot.  These  lower  percentages  were  caused  not  by 
lack  of  participation,  but  by  technical  difficulties  causing  a  loss  of  data. 

MMPl-2  data  fell  within  clinical  means.  MAB  findings  confirmed  a  superior 
level  of  cognitive  functioning  among  this  group.  A  curious  finding  was  the  elevated 
means  on  Arithmetic,  Vocabulary,  and  [Digit  Symbol  subtests.  One  might  expect 
spatial  skills  to  be  in  this  elevated  category  since  right  brain  skills  aie  often 
considered  obligatory  in  flying  (24).  Likewise,  the  rapid  assimilation  of  data  needed 
to  fly  missions  in  changing  environments  might  be  ref.ected  in  high  mean 
comprehension  scores.  The  elevations  on  Arithmetic,  Vocabulary  and  Digit  Symbol 
subtests  suggests  high  left  brain  skill  levels  with  greater  attention  to  detail  and 
math/language  skills. 

PCI  data  demonstrated  a  range  of  groups  in  this  squadron;  80%  fall  within 
the  best  and  middle  groups,  and  20%  into  what  is  considered  the  poor  group. 
No  pilot  scored  in  the  worst  group  category.  Deployed  in  one  or  more  pairs  of 
aircraft,  it  is  sometimes  forgotten  that  single-seat  fighter  pilots  still  need  crew 
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.asource  management  skills.  Communication  between  pilots,  and  then  delegation 
and  division  of  tasks  are  critical  mission  duties.  While  the  results  of  the  top 
performer  survey  confirmed  more  lead  pilots  (4)  chosen  from  the  best  group,  one 
lead  was  chosen  from  the  poor  category  on  the  PCI. 

Picano  (18)  has  suggested  that  there  is  no  “one  type  of  personality’  identified 
as  best  for  military  pilot  selection.  While  pilot  groups  may  be  homogeneous  in 
certain  characteristics,  they  still  have  areas  of  uniqueness  and  diversity.  While 
our  data  doesn’t  address  selection  for  pilot  training,  it  does  support  the  premise 
that  individuals  chosen  to  be  in  a  combat  flight  chow  a  variety  of  qualities.  In 
short,  it  takes  all  different  kinds  of  individuals  to  fly  and  complete  the  mission. 

Squadron  members  did  tend  to  agree  about  two  of  their  top  performers, 
choosing  two  individuals  for  lead  and  one  for  wing  with  a  majority  (62%,  44%, 
and  12%,  respectively)  of  their  votes.  Leadership,  job  performance,  aggressiveness 
and  knowledge  were  the  most  frequently  chosen  characteristics  for  lead  position. 
Job  performance,  knowledge,  teamwork,  and  personality  were  the  most  frequently 
chosen  characteristics  for  wing  position.  This  battery  can  gather  important  data 
from  a  sample  as  varied  as  the  squadrons  that  fly  the  missions.  With  aviator 
volunteers,  aircrew  qualities  of  top  performers  can  be  studied. 

Finally,  the  question  of  correlations  between  top  performers  and  their  testing 
iesults  remains  to  be  answered.  This  testing  protocol  was  a  successful  approach 
in  obtaining  data  from  a  normative  aircrew  sample.  With  minor,  but  necessary 
refinements:  it  could  be  widely  implemented  for  a  range  of  aircrew  in  varying 
airframes.  Once  a  larger  sample  can  be  studied,  attempts  can  be  made  to  define 
desirable  personal  qualities  and  testing  profiles.  This  project  has  successfully 
produced  a  format  that  can  obtain  the  information. 


Recommendations  for  Future  Research 

There  are  multiple  payoffs  from  this  test  battery.  Psychometric  norms  could 
be  updated,  and  aircraft  specific  norms  could  be  developed.  USAF  ACS  waiver 
recommendations  would  benefit  from  normative  data.  Baseline  cognitive 
measurements  on  aviators  could  be  obtained.  Gathering  information  about  aviators 
throughout  their  careers  could  yield  important  information  about  retention.  Primary 
prevention  of  mental  health  difficulties  would  benefit  aircrew  (and  mission) 
performance,  through  data  obtained  with  the  C-DIS. 

Finally,  with  sufficient  aircrew  participation,  attempts  could  be  made  to  correlate 
top  performers’  psychological  profiles  with  their  personal  qualities  (as  seen  by 
associates).  Data  from  this  battery  could  direct  training  to  prepare  future  flight 
leads.  Current  USAF  research  into  situational  awareness,  a  skill  thought  to  be 
highly  related  to  combat  survivability,  could  gain  from  information  about  top 
performer  qualities  and  psychological  profiles. 

As  the  USAF  adapts  to  face  the  challenges  of  the  21st  century,  one  thing 
will  remain  constant.  The  success  of  the  mission  will  depend  upon  the  capabilities 
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and  performance  of  the  aviator.  Payoffs  follow  the  search  for  understanding  of 
the  well-adapted  aircrew  member.  This  test  battery  can  gather  information  that 
might  aid  in  training  the  successful  pilot  of  the  future.  The  concept  of  training 
cockpit  resource  management  in  aviators  is  one  important  example  of  the  interface 
between  psychological  skills  and  effective  mission  performance.  By  identifying 
those  qualities  of  successful  pilots  and  those  qualities  needed  for  lead  and  wing 
positions,  the  data  (personality,  intelligence,  interpersonal  qualities,  peer  ratings 
and  incidence  of  psychiatric  diagnosis)  can  tell  future  managers  who  to  retain  in 
a  downsized  armed  force. 
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APPENDIX  A 
PEER  SURVEY 
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RATING  CATEGORIES 


1.  GENERAL  KNOWLEDGE 

♦possesses  a  good  fund  of  information 
♦absorbs  new  information  quickly 
♦reduces  complex  issues  to  essential  elements 
♦valued  for  his/her  opinions  on  technical  matters 

2.  JOB  PERFORMANCE 

♦accomplishes  any  task  thoroughly  and  efficiently 
♦uses  initiative  to  solve  difficult  problems 
♦is  predictable,  consistent,  reliable  in  performance 
♦able  to  prioritize  multiple  critical  tasks  quickly 

3.  STRESS  TOLERANCE 

♦demonstrates  prompt  and  accurate  reactions 
♦effective  in  an  unexpected  emergency 
♦effective  under  prolonged  periods  of  stress 
♦arrives  at  practical  conclusions  in  emergencies 

4.  LEADERSHIP 

♦motivates  others  to  complete  tasks 

♦delegates  work  and  allows  person  to  complete  task 

♦is  decisive/flexible  when  r^uired 

♦has  determination  and  projects  decisiveness 

5.  GROUP  COHESIVENESS 

♦puts  group  goals  ahead  of  individual  goals 
♦shares  cred'tt  and  accepts  blame 
♦tolerant  of  individual/cultural  differences 
♦works  effectively  with  many  different  people 

6.  TEAMWORK 

♦easy  to  get  along  with,  good  sense  of  humor 
♦pulls  own  weight  (does  own  share  of  undesirable  tasks) 
♦gives  euid  accepts  foedback/criticism  well 
♦good  listener 

7.  PERSONAUTY 

♦tolerates  difficulties  and  frustration  well 
♦few  irritating  qualities 
♦personable  and  amiable 
♦self-sufficient,  motivated,  self-starter 

8.  COMMUNICATION  SKILLS 

♦presents  seif  well;  speaks  clearly  and  effectively 
♦represents  squadron  well 
♦concise  and  focused 
♦gets  point  across 

9.  AGGRESSIVENESS 

♦pursues  goals  rather  than  waiting  for  them  to  occur 
♦accepts  calculated  risks 
♦“makes"  opportunities  where  few  seem  to  exist 
♦desire  to  excel 
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PEER  SURVEY 


Choose  from  your  squadron  those  three  other  pilots  you  would  want  fill  your 
four-ship  for  combat,  and  place  their  names  in  the  spaces  below.  Considering 
those  qualities  on  the  facing  page,  name  those  that  were  most  important  in  your 
choices  and  rank  these  in  the  spaces  provided  beneath  each  name  (most  important 
1,  next  most  important  2,  etc.  3,  4). 

Remember  that  we  are  looking  for  the  “best*  person  you  feel  will  fill  each 
job,  but  DO  NOT  choose  yourself. 


Lead: 


1st  Choice  Wingman:  2nd  Choice  Wingman: 


(  name  )  (  name  )  (  name  ) 


(  Trustee:  remove  top  portion  of  this  form  and  destroy  ] 
What  qualities  were  most  important  in  choosing  this  person? 


#1. 

#1. 

#1. 

#2. 

#2. 

#2. 

#3. 

#3. 

#3. 

#4. 

#4. 

#4. 

Trustee  use  only  for  random  number  assignments: 


Aircrew  member:  Trustee  will  simply  separate  names  from  top  of  page  and 
send  bottom  of  page  with  assigned  random  numbers  to  principal  investigator. 


27 


APPENDIX  B 
CONSENT  FORM 


29 


VOLUNTARY  CONSENT  STATEMENT 


DEVELOPMENT  OF  A  COMPUTER  DRIVEN  TOOL  TO 
MEASURE  PSYCHOLOGICAL  CHARACTERISTICS  OF  AIRCREW 

1.  I  understand  the  Neuropsychiatry  Branch  of  the  Armstrong  Laboratory.  Brooks  AFB,  TX  and  I27th  TAC 
Clinic,  Selfridge  ANG,  Ml  are  conducting  a  research  study  of  norms  of  psychological  profiles  of  well  adapted 
and  well  functioning  aircrew.  The  purpose  of  this  study  is  to  develop  a  perspective  of  the  “successful” 
aviator  and  gather  normative  data  with  a  screening  battery  of  psychometric  measures.  My  participation  in 
this  study  will  consist  of  taking  a  battery  of  four  psychometric  tests  on  a  computer  taking  approximately  5 
hours  of  total  time  divided  in  two  equal  sessions,  on  two  separate  days. 

2.  I  agree  to  truthfully  answer  a  variety  of  questions  from  four  standardized  psychological  tests.  I  will 
select  random  numbers  to  attach  to  my  disks  and  will  send  my  name  and  random  numbers  in  a  sealed 
envelope  to  a  third  disinterested  party  chosen  by  the  squadron  aircrew.  I  will  then  fill  out  the  “four  ship 
formation”  survey  identifying  my  preferences  of  pilots  for  flying  into  combat.  I  will  send  my  survey  in  a 
separate  sealed  envelope  to  a  disinterested  third  party  where  I  understand  my  answers  will  be  held  In 
anonymity.  The  third  party  will  translate  names  by  the  key  into  random  number  codes,  then  destroy  the 
top  half  of  the  survey.  The  trustee  will  destroy  the  key  after  coding  the  surveys. 

3.  There  are  no  anticipated  hazards  or  discomfort  that  I  may  experience  from  my  participation  in  this  study. 

4.  I  may  not  receive  direct  benefit  from  this  research,  but  the  benefits  of  my  participation  is  helping  develop 
psychological  test  norms  for  typical,  healthy  aircrew.  I  understand  I  will  not  receive  individual  feedback  on 
my  test  results. 

5.  There  are  no  other  equivalent  alternative  procedures  to  obtain  the  data  sought  by  this  research  protocol 

6.  Records  of  my  participation  in  this  study  may  only  be  disclosed  according  to  federal  law,  including  the 
Federal  Privacy  Act,  5  U.S.C.  552a,  and  its  implementing  regulations.  I  understand  that  every  possible 
effort  will  be  made  to  protect  my  anonymity.  My  results  will  be  stored  under  a  random  number  which  I 
select.  The  random  number  will  be  removed  from  my  disk  that  would  normally  allow  results  to  be  traced 
to  my  name  immediately  after  the  data  compilation.  The  data  will  only  be  accessible  by  group  results  and 
not  by  individual  data. 

7.  I  understand  that  my  entitlements  to  medical  care  or  compensation  in  the  event  of  injury  are  governed 
by  federal  laws  and  regulations,  and  if  I  desire  further  information,  I  may  contact  the  Brooks  AFB  Legal 
Office  or  the  principal  investigator. 

8.  The  decision  to  participate  in  this  research  is  completely  voluntary  on  my  part.  No  one  has  coerced 
or  intimidated  me  into  participating  in  this  program.  I  am  participating  because  I  want  to.  Major  Flynn, 
Major  Sipes  or  LTC  Grosenbach  has  adequately  answered  any  and  all  questions  I  have  about  this  study, 
my  participation  and  the  procedures  involved.  I  understand  Major  Flynn,  Major  Sipes  or  LTC  Grosenbach 
will  be  available  to  answer  any  questions  concerning  procedures  throughout  this  study.  I  understand  that 
if  significant  new  findings  develop  during  the  course  of  this  research  which  may  relate  to  my  decision  to 
continue  participation,  I  will  be  informed.  I  further  understand  that  I  may  withdraw  this  consent  at  any  time 
and  discontinue  further  participation  in  this  study  without  prejudice  to  my  entitlements.  I  also  understand 
that  the  medical  monitor  of  this  study  may  terminate  my  participation  in  this  study  if  he  or  she  feels  this 
to  be  in  my  best  interest,  or  the  best  interest  of  the  government. 


Volunteer  Signature  and  Social  Security  Number 

Date 

Witness  1  (not  directly  involved)  Date 

Witness  2  (not  directly  involved) 

Date 

Principal  Investigator 

Date 
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